OCR configuration for a Canofy customer

1. How OCR authentication works

1. The customer's Frappe site sends POST /generate_ocr to the OCR server with a username, a password and the S3 path of the PDF.
2. The OCR server looks up the caller's private IP in property.json. It decrypts that entry's username and password with the ENCRYPTION_KEY from .env (Fernet) and compares them with what was sent.
3. If they match, it reads the PDF from the entry's bucket and returns the extracted data.

Each step is tagged with where it runs:

OCR server, as ubuntu Server via SSM, as frappe Server via SSM, as ssm-user AWS console AWS CloudShell Canofy in the browser

Warning: the username and password must be the same plain values on both sides. They're encrypted (Fernet) in property.json, and base64-encoded in site_config.json.

Values for this customer

2. Prerequisites in AWS

2.1 OCR access to the customer bucket

AWS console IAM → Policies → OCR-production-S3 → Edit → JSON. Add these lines to both the Allow statement and the Deny/NotResource statement, right after a line that already ends with a comma:

"arn:aws:s3:::bel.canofy.tech",
"arn:aws:s3:::bel.canofy.tech/*",

Then Next → Save changes. Confirm in CloudShell that the bucket appears 4 times:

AWS CloudShell

ARN=$(aws iam list-policies --scope Local --query "Policies[?PolicyName=='OCR-production-S3'].Arn" --output text); VER=$(aws iam get-policy --policy-arn "$ARN" --query Policy.DefaultVersionId --output text); aws iam get-policy-version --policy-arn "$ARN" --version-id "$VER" --query PolicyVersion.Document --output json | grep -c "bel.canofy.tech"

2.2 Network: port 5000

AWS console EC2 → OCR server → Security → security group → Edit inbound rules → Add rule: Custom TCP, port 5000, source 172.31.10.27/32, description BEL bel.canofy.tech.

Server via SSM, as ssm-user Test from the customer server. Any HTTP code (400/401/500) means it's reachable; 000 means it's blocked.

curl -s -m 5 -o /dev/null -w "%{http_code}\n" -X POST http://172.31.35.206:5000/generate_ocr -H "Content-Type: application/json" -d "{}"

3. OCR server: find the live files

OCR server, as ubuntu Connect to the OCR server through SSM, then confirm the service and its working directory:

systemctl status ocr_app.service --no-pager | head -5

systemctl cat ocr_app.service | grep -E "WorkingDirectory|ExecStart"

Several copies of property.json exist. Only the one in the WorkingDirectory is live:

sudo find / \( -iname "property.json" -o -iname "properties.json" \) -not -path "/proc/*" -not -path "*/node_modules/*" 2>/dev/null

To confirm which path the code opens:

sudo grep -rn "property.json" /home/ubuntu/canofy_OCR/*.py

A relative "property.json" means the file in the WorkingDirectory.

Switch to the file owner. As ssm-user, you can't read /home/ubuntu:

sudo -iu ubuntu

Back up before any change:

cp ~/canofy_OCR/property.json ~/canofy_OCR/property.json.bak.$(date +%Y%m%d-%H%M) && ls -la ~/canofy_OCR/property.json*

grep -c "^ENCRYPTION_KEY=" ~/canofy_OCR/.env

The last command must print 1.

4. Update property.json

Choose one of the two methods below. Both use the existing key, generate a random per-customer username and password, and print the three Frappe commands needed in section 5.

4.1 Method A: helper script (recommended)

AWS console Upload ocr_add_customer.py (appendix A) to s3://bel.canofy.tech/_setup/. The OCR role can read that bucket.

OCR server, as ubuntu As ubuntu:

aws s3 cp s3://bel.canofy.tech/_setup/ocr_add_customer.py ~/ocr_add_customer.py --region ap-south-1

Dry run (it lists the existing IP → bucket entries and checks they all decrypt):

/usr/bin/python3 ~/ocr_add_customer.py --app-dir /home/ubuntu/canofy_OCR --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech --ocr-url http://172.31.35.206:5000/generate_ocr --dry-run

Real run (it backs up, writes and validates the file, then prints the Frappe commands):

/usr/bin/python3 ~/ocr_add_customer.py --app-dir /home/ubuntu/canofy_OCR --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech --ocr-url http://172.31.35.206:5000/generate_ocr

It refuses to overwrite an IP that already exists. To rotate an existing customer's credentials, add --replace.

4.2 Method B: manual

Step 1: generate the values. Keep this terminal open, and don't share its output:

cd ~/canofy_OCR && /usr/bin/python3 -c "import secrets,base64;from cryptography.fernet import Fernet;k=[l.split('=',1)[1].strip().strip('\"') for l in open('.env') if l.startswith('ENCRYPTION_KEY=')][0];f=Fernet(k.encode());u='bel_ocr_'+secrets.token_hex(3);p=secrets.token_urlsafe(24);print('ENC_USERNAME =',f.encrypt(u.encode()).decode());print('ENC_PASSWORD =',f.encrypt(p.encode()).decode());print(' bench --site bel.canofy.tech set-config ocr_username',base64.b64encode(u.encode()).decode());print(' bench --site bel.canofy.tech set-config ocr_password',base64.b64encode(p.encode()).decode());print(' bench --site bel.canofy.tech set-config ocr_url http://172.31.35.206:5000/generate_ocr')"

For a new customer, change bel_ocr_ and bel.canofy.tech in this command.

Step 2: check the format of the existing entries. Look at whether values are plain "gAAAA…" or "b'gAAAA…'", and use the same style:

head -c 400 ~/canofy_OCR/property.json; echo

Step 3: edit the file.

nano ~/canofy_OCR/property.json

Add a comma after the last entry's closing }, then this block. Paste with right-click, save with Ctrl+O and Enter, exit with Ctrl+X.

"172.31.10.27": {
    "BUCKET NAME": "bel.canofy.tech",
    "username": "PASTE_ENC_USERNAME",
    "password": "PASTE_ENC_PASSWORD"
}

If the SSM terminal merges pasted lines into one, that's fine: JSON ignores line breaks. Only the commas, quotes and braces matter.

4.3 Validate (both methods)

This prints only IPs, buckets and True/False:

cd ~/canofy_OCR && /usr/bin/python3 -c "import json;from cryptography.fernet import Fernet;k=[l.split('=',1)[1].strip().strip('\"') for l in open('.env') if l.startswith('ENCRYPTION_KEY=')][0];f=Fernet(k.encode());d=json.load(open('property.json'));[print(ip,'->',v.get('BUCKET NAME'),'| decrypts:',all(bool(f.decrypt(str(v[x]).strip('b').strip(\"'\").encode())) for x in ('username','password'))) for ip,v in d.items()]"

Expected: every customer listed, including 172.31.10.27 -> bel.canofy.tech | decrypts: True.

Show the plain username of the new entry, to compare with Frappe in section 5.3:

cd ~/canofy_OCR && /usr/bin/python3 -c "import json;from cryptography.fernet import Fernet;k=[l.split('=',1)[1].strip().strip('\"') for l in open('.env') if l.startswith('ENCRYPTION_KEY=')][0];f=Fernet(k.encode());v=json.load(open('property.json'))['172.31.10.27'];print(f.decrypt(v['username'].strip('b').strip(\"'\").encode()).decode())"

If anything fails, restore the latest backup:

cp "$(ls -t ~/canofy_OCR/property.json.bak.* | head -1)" ~/canofy_OCR/property.json

4.4 Restart the OCR service

The service is shared, so restart it at a quiet moment:

sudo systemctl restart ocr_app.service && sudo systemctl status ocr_app.service --no-pager | head -5

Expected: active (running).

5. Update the Frappe site config

Server via SSM, as frappe Connect to the customer server through SSM, then:

sudo -iu frappe

cd ~/frappe-bench

5.1 Set the three keys

Paste the three lines printed in section 4. They begin with a space, so they stay out of shell history:

 bench --site bel.canofy.tech set-config ocr_username 'BASE64_USERNAME'

 bench --site bel.canofy.tech set-config ocr_password 'BASE64_PASSWORD'

bench --site bel.canofy.tech set-config ocr_url 'http://172.31.35.206:5000/generate_ocr'

If you only have the plain values, encode them yourself. Use printf '%s', not echo, because echo adds a newline that breaks the match:

 printf '%s' 'PLAIN_OCR_USERNAME' | base64

 printf '%s' 'PLAIN_OCR_PASSWORD' | base64

5.2 Check (values masked)

python3 -c "import json;d=json.load(open('sites/bel.canofy.tech/site_config.json'));[print(k,'=',('*****' if k!='ocr_url' else d[k])) for k in ('ocr_url','ocr_username','ocr_password') if k in d]"

Expected:

ocr_url = http://172.31.35.206:5000/generate_ocr
ocr_username = *****
ocr_password = *****

Or use Frappe's own view:

bench --site bel.canofy.tech show-config | grep -E "ocr_url|host_name"

5.3 Check that the username matches property.json

python3 -c "import json,base64;d=json.load(open('sites/bel.canofy.tech/site_config.json'));print(base64.b64decode(d['ocr_username']).decode())"

This must print exactly the same username as the check in section 4.3.

5.4 Apply

bench restart

5.5 Other config commands

6. Test end to end

6.1 Find a real PDF in the customer bucket

Server via SSM, as frappe

aws s3 ls s3://bel.canofy.tech/BEL/ --recursive --region ap-south-1 | grep -i "\.pdf" | tail -5

Note the key (the path after the date, size and space).

6.2 Call OCR directly with the site's own config

This sends the same credentials the site uses, so it separates configuration problems from application problems. Replace BEL/your/file.pdf with the key from 6.1:

cd ~/frappe-bench && ./env/bin/python -c "import json,base64,requests;c=json.load(open('sites/bel.canofy.tech/site_config.json'));r=requests.post(c['ocr_url'],json={'username':base64.b64decode(c['ocr_username']).decode(),'password':base64.b64decode(c['ocr_password']).decode(),'filename':'BEL/your/file.pdf'},timeout=300);print(r.status_code);print(r.text[:600])"

The payload format (username, password, filename) follows the OCR App Setup Guide. If the Canofy app sends it differently, compare with canofy_tcs/import_integrations/doctype/boe_bridge/boe_bridge.py.

6.3 Test from the application, and read the OCR log

Canofy in the browser Run OCR on a document the way users do, for example from a Bill of Entry or Icegate Intimation. Then:

OCR server, as ubuntu

sudo journalctl -u ocr_app.service --since "15 minutes ago" --no-pager | tail -80

7. Change, rotate or remove a customer

Every OCR-server change follows the same pattern: back up → edit → validate (4.3) → restart (4.4).

8. Troubleshooting

9. Security rules

• Never change ENCRYPTION_KEY. It protects every customer's entry.
• The OCR App Setup Guide currently contains the real key in plain text. Replace it with a placeholder in that document, and plan a key rotation (re-encrypt all entries in one change).
• Never paste the plain or encrypted credentials into chat, tickets or email; keep them in the password manager.
• Keep the property.json.bak.* files on the server; don't copy them elsewhere.
• Delete ocr_add_customer.py from _setup/ in the customer bucket after use, and keep the master copy in the DevOps repository.

Appendix

A. ocr_add_customer.py

Helper for section 4.1. Run on the OCR server as ubuntu with /usr/bin/python3. It reads the key from .env and never prints it.

#!/usr/bin/env python3
"""
Canofy OCR - add a customer (ERP server) to property.json

Run ON THE OCR SERVER, with the same Python the OCR service uses:

  /usr/bin/python3 ocr_add_customer.py --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech --dry-run
  /usr/bin/python3 ocr_add_customer.py --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech

What it does
  * reads ENCRYPTION_KEY from ~/canofy_OCR/.env (never typed, never printed)
  * generates a random OCR username + password for this customer
  * Fernet-encrypts them exactly like encrypt.py, and checks they decrypt
  * refuses to overwrite an existing IP entry (use --replace to rotate)
  * backs up property.json, writes the new entry, validates the JSON
  * prints the 3 'bench set-config' commands to run on the ERP server
    (ocr_username / ocr_password base64-encoded, ocr_url)
"""
import argparse
import base64
import json
import os
import secrets
import shutil
import socket
import sys
from datetime import datetime

from cryptography.fernet import Fernet


def read_env_key(env_path):
    with open(env_path) as f:
        for line in f:
            line = line.strip()
            if line.startswith("ENCRYPTION_KEY="):
                return line.split("=", 1)[1].strip().strip('"').strip("'")
    sys.exit(f"ERROR: ENCRYPTION_KEY not found in {env_path}")


def own_private_ip():
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        s.connect(("10.255.255.255", 1))
        return s.getsockname()[0]
    finally:
        s.close()


def main():
    home = os.path.expanduser("~")
    p = argparse.ArgumentParser()
    p.add_argument("--code", required=True, help="customer code, e.g. BEL")
    p.add_argument("--ip", required=True, help="ERP server PRIVATE IP, e.g. 172.31.10.27")
    p.add_argument("--bucket", required=True, help="customer S3 bucket, e.g. bel.canofy.tech")
    p.add_argument("--site", required=True, help="Frappe site name, e.g. bel.canofy.tech")
    p.add_argument("--app-dir", default=os.path.join(home, "canofy_OCR"))
    p.add_argument("--ocr-url", help="default: http://<this server private IP>:5000/generate_ocr")
    p.add_argument("--replace", action="store_true", help="rotate credentials of an existing IP entry")
    p.add_argument("--dry-run", action="store_true")
    a = p.parse_args()

    prop_path = os.path.join(a.app_dir, "property.json")
    env_path = os.path.join(a.app_dir, ".env")
    for path in (prop_path, env_path):
        if not os.path.isfile(path):
            sys.exit(f"ERROR: not found: {path}  (use --app-dir if the OCR app lives elsewhere)")

    fernet = Fernet(read_env_key(env_path).encode())
    with open(prop_path) as f:
        props = json.load(f)
    if not isinstance(props, dict):
        sys.exit("ERROR: property.json is not a JSON object")

    # Report existing entries (IP -> bucket) and sanity-check their format
    print(f"property.json has {len(props)} entries:")
    for ip, entry in props.items():
        print(f"  {ip:16} -> {entry.get('BUCKET NAME')}")
        u = str(entry.get("username", ""))
        if u.startswith("b'") or u.startswith('b"'):
            print("    NOTE: this entry stores values as b'...' text; new entry uses plain token text")
        try:
            fernet.decrypt(u.strip("b'\"").encode())
        except Exception:
            print("    WARNING: this entry does not decrypt with the current key")

    if a.ip in props and not a.replace:
        sys.exit(f"ERROR: {a.ip} already exists (bucket {props[a.ip].get('BUCKET NAME')}). Use --replace to rotate.")
    for ip, entry in props.items():
        if entry.get("BUCKET NAME") == a.bucket and ip != a.ip:
            print(f"WARNING: bucket {a.bucket} is already used by {ip}")

    username = f"{a.code.lower()}_ocr_{secrets.token_hex(3)}"
    password = secrets.token_urlsafe(24)
    enc_user = fernet.encrypt(username.encode()).decode()
    enc_pass = fernet.encrypt(password.encode()).decode()
    assert fernet.decrypt(enc_user.encode()).decode() == username
    assert fernet.decrypt(enc_pass.encode()).decode() == password

    new_entry = {"BUCKET NAME": a.bucket, "username": enc_user, "password": enc_pass}
    ocr_url = a.ocr_url or f"http://{own_private_ip()}:5000/generate_ocr"

    print(f"\nNew entry for {a.ip}: BUCKET NAME={a.bucket}, username/password = Fernet tokens (verified)")
    if a.dry_run:
        print("\nDRY RUN - property.json not changed.")
        return

    backup = f"{prop_path}.bak.{datetime.now():%Y%m%d-%H%M%S}"
    shutil.copy2(prop_path, backup)
    props[a.ip] = new_entry
    tmp = prop_path + ".tmp"
    with open(tmp, "w") as f:
        json.dump(props, f, indent=4)
    with open(tmp) as f:
        json.load(f)  # validate before replacing
    os.replace(tmp, prop_path)
    print(f"Backup : {backup}")
    print(f"Written: {prop_path}  ({len(props)} entries)")

    b64 = lambda s: base64.b64encode(s.encode()).decode()
    print("\n=========== RUN THESE ON THE ERP SERVER (as frappe, in ~/frappe-bench) ===========")
    print(f"bench --site {a.site} set-config ocr_username {b64(username)}")
    print(f"bench --site {a.site} set-config ocr_password {b64(password)}")
    print(f"bench --site {a.site} set-config ocr_url {ocr_url}")
    print("===================================================================================")
    print("These are credentials: copy them straight into the ERP session. Do not paste them")
    print("into chat, tickets or email. Then restart the OCR service:")
    print("  sudo systemctl restart ocr_app.service && sudo systemctl status ocr_app.service --no-pager")


if __name__ == "__main__":
    main()

B. Screenshot register

Figures from the Belimo setup, with sensitive identifiers masked, and screenshots still to capture for the SOP.